o 



(19) 



J) 



EuropSisches Pateniamt 
European Patent Office 
Office europeen des brevets 




(11) 



EP 0 891 071 A2 



(12) 



EUROPEAN PATENT APPLICATION 





Date of publication: 


(51) IntCI. 6 : H04N 1/32 




13.01.1999 Bulletin 1999/02 


(21) 


Application number: 98112561.0 




(22) 


Date of filing: 07.07.1998 




(84) 


Designated Contracting States: 


(72) Inventor: Senoh, Takanori 




AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 


Hirakata-shi, Osaka (JP) 




MCNL PTSE 




Designated Extension States: 


(74) Representative: 




AL LT LV MK RO SI 


Schwabe - Sandmair - Marx 


(30) 




Stuntzstrasse 16 


Priority: 09.07.1997 JP 183429/97 


81677 Munchen (DE) 


(71) 


Applicant: 






Matsushita Electric Industrial Co., Ltd. 






Kadoma-shi, Osaka 571 -8501 (JP) 





CM 
< 



(54) Apparatus and method for watermark data insertion and apparatus and method for 
watermark data detection 



(57) A watermark data insertion method for insert- 
ing watermark data into an input original signal is pro- 
vided. The method includes: a frequency transform step 
for applying a frequency transform to the original signal 
to form an intermediate signal; a watermark data inser- 
tion stop for inserting the watermark data into a first set 
of frequency component of the intermediate signal; and 
a frequency inverse transform stop for applying an 
inverse frequency transform to the intermediate signal, 
the watermark data having been inserted into the inter- 

FIG. 1 

2- 



mediate signal, thereby obtaining a signal containing 
the watermark data embedded therein. The watermark 
data insertion step includes: determining, based on 
pseudo random numbers, the first set of frequency com- 
ponents of the intermediate signal into which the water- 
mark data is inserted; and using a second set of 
frequency components of the intermediate signal as a 
reference signal, the watermark data not being inserted 
into the second set of frequency components. 
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Description 

BACKGROUND OF THE INVENTION 

1. FIELD OF THE INVENTION: 

The present invention relates to a watermark dots 
insertion method for inserting watermark data into a 
digitaiized signal representing images, sounds, and/or 
the like to indicate their copyrights, and a watermark 
data detection method for detecting inserted watermark 
data. 

2. DESCRIPTION OF THE RELATED ART: 

Techniques for inserting watermark data into a digi- 
talized signal representing images, sounds, and/or the 
like to indicate their copyrights having been vigorously 
studied and developed. Such watermark data is 
required to have minimum interference with the images 
and/or sounds of interest, be hardly visible (in the case 
of image data) and hardly audible (in the cane of sound 
data), and yet be easily detected by a person knowing 
the detection method while preventing unauthorized 
removal of the watermark data by third parties. By using 
such watermark data, for example, infringement of a 
copyright (by illegal copying of images and/or sounds) 
by unauthorized third parties can be established or pre- 
vented. 

One conventional method for inserting watermark 
data into an original signal representing images, 
sounds, and/or the like utilizes a spread spectrum tech- 
nology. In principle; this method involves: multiplying the 
original signal representing images, sounds, and/or the 
like by random numbers to obtain a uniform spectrum; 
adding watermark data to the multiplied signal; and 
again multiplying the resultant signal with the same ran- 
dom numbers to restore the original signal. At the time 
of restoring the original signal, the added watermark 
data is also multiplied by the random numbers. As a 
result, the watermark data has a uniformly spread spec- 
trum, i.e. becomes white noise, in the restored original 
signal. 

Figure 7 is a block diagram illustrating the structure 
of a conventional watermark data insertion apparatus 
utilizing a spread spectrum technology. As shown in Fig- 
ure 7, an original signal a t (into which watermark data Xj 
is to be inserted) is input to a multiplier 52 p where it is 
multiplied by predetermined pseudo random numbers 
to become a^j. Since the pseudo random numbers Cj 
are a random sequence of +1 and -1, the frequency 
spectrum of B^c t becomes uniform. 

Next, the watermark data x s is input to an adder 53 
to be added to a DC component of the above multiplica- 
tion product ajXCj. Although not shown, the adder 53 
includes a cascade connection of a Fourier transformer, 
an adder, and an inverse Fourier transformer for extract- 
ing a DC component of an input signal. 



The signal ajXCj + Xj (i.e., the randomized original 
signal plus the watermark data) is input to a multiplier 
54, where it is again multiplied by the same pseudo ran- 
dom numbers to give a^j 2 + XjxCj. Since the pseudo 

5 random numbers are +1 or -1 , q 2 equals 1 . Therefore, a 
signal having a value of a; + XjXCj (= ajxc 2 + XjXCj) is 
obtained, i.e., a signal with embedded watermark data. 

Figure 8 is a block diagram illustrating the structure 
of a conventional watermark data detection apparatus 

io utilizing a spread spectrum technology for detecting 
embedded watermark data. As shown in Figure 8, a sig- 
nal with embedded watermark data a t + XjXCj is input to 
a multiplier 62, where it ii multiplied by predetermined 
pseudo random numbers to become ajXCj + XjXc 2 = 

is a s xCi + Xj. Since the pseudo random numbers ci are a 
uniform random sequence of +1 and -1, the DC compo- 
nent of ajXCj becomes zero, so that only the watermark 
data xj is extracted by an extractor 63. The extractor 63 
includes a Fourier transformer for extracting a DC com- 

20 ponent of an input signal. 

The above-described technique is reported in 
"Watermark data signature method for images utilizing 
PN series", Electronic Information Communication 
Society "Encryption and Information Security Sympo- 

25 siurrf proceedings. SCIS *97-26B, 1997. 

However, the above-described conventional tech- 
nique has a disadvantage in that the watermark data 
may become noticeable where the original signal has a 
small amplitude because the watermark data is inserted 

30 into the original signal in a uniform manner, i.e., regard- 
less of the characteristics of the original signal. There is 
also a disadvantage in that, if the amplitude of the 
watermark data is decreased in order to prevent the 
watermark data from becoming noticeable, the water- 

35 mark data may be masked by the original signal and 
therefore difficult to detect. Furthermore, unauthorized 
third parties can make the watermark data impossible to 
detect by intentionally adding noise or the like to the 
original signal. 

40 

SUMMARY OF THE INVENTION 

A watermark data insertion method for inserting 
watermark data into an input original signal according to 

45 the present invention includes: a frequency transform 
step for applying a frequency transform to the original 
signal to form an intermediate signal; a watermark data 
insertion step for inserting the watermark data into a 
first set of frequency components of the intermediate 

so signal; and a frequency inverse transform step for 
applying an inverse frequency transform to the interme- 
diate signal, the watermark data having been inserted 
into the intermediate signal, thereby obtaining a signal 
containing the watermark data embedded therein. 

55 wherein the watermark data insertion step includes: 
determining, based on pseudo random numbers, the 
first set of frequency components of the intermediate 
signal into which the watermark data in inserted; and 
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using a second set of frequency components of the 
intermediate signal as a reference signal, the water- 
mark data not being inserted into the second set of fre- 
quency components. 

In one embodiment of the invention, the frequency 
transform applied in the frequency transform step is a 
wavelet transform, and the inverse frequency transform 
applied in the frequency inverse transform step is an 
inverse wavelet transform. 

In another embodiment of the invention, the water- 
mark data insertion step includes: a weighting step for 
increasing or decreasing a signal amplitude of the 
watermark data in accordance with an amplitude of the 
first set of frequency components of the intermediate 
signal into which the watermark data is inserted. 

In still another embodiment of the invention, the 
watermark data insertion step includes determining the 
first set of frequency components of the intermediate 
signal into which the watermark data is inserted by 
using pseudo random numbers having frequency- 
dependent weights. 

In still another embodiment of the invention, the 
watermark data insertion step includes: a weighting 
step for increasing or decreasing a signal amplitude of 
the watermark data in accordance with a frequency of 
the first sat of frequency components of the intermedi- 
ate signal into which the watermark data is inserted. 

In still another embodiment of the invention, the 
watermark data insertion step includes converting 
binary data representing the watermark data into prede- 
termined positive and negative values, and thereafter 
inserting the converted binary data into the first set of 
frequency components of the intermediate signal. 

Alternatively, the watermark data insertion method 
for inserting watermark data into an input original signal 
according to the present invention includes: determining 
a position at which the watermark data is to be inserted 
in accordance with pseudo random numbers which 
have been subjected to an inverse transform; subjecting 
the watermark data to an inverse transform; increasing 
or decreasing a signal amplitude of the inverse-trans- 
formed watermark data in accordance with an ampli- 
tude of the original signal; and inserting the inverse- 
transformed watermark data with the increased or 
decreased signal amplitude into the original signal. 

A watermark data detection method for detecting 
watermark data within a signal containing embedded 
watermark data according to the present invention 
includes: an inverse-transformed pseudo random 
number generation stop for applying an inverse fre- 
quency transform to pseudo random numbers used for 
determining an insertion position of the watermark data 
and complements of the pseudo random numbers, 
respectively; a multiplication step for multiplying the sig- 
nal containing embedded watermark data by the values 
generated in the Inverse-transformed pseudo random 
number generation step to obtain signal components to 
which the watermark data has been added and signal 



components to which the watermark data has not been 
added; and an averaging stop for deriving a difference 
between an average of the signal components to which 
the watermark data has been added and an average of 
5 the signal components to which the watermark data has 
not been added as obtained in the multiplication step, 
whereby the difference is detected as the watermark 
data. 

In one embodiment of the invention, the averaging 
w step includes deriving the average of the signal compo- 
nents to which the watermark data has been added by 
applying a weight to an average of high frequency com- 
ponents among the signal components to which the 
watermark data has been added, the weight being 
is smaller than weights applied to averages of frequency 
components other than the high frequency components. 

Alternatively, the watermark data detection method 
for detecting watermark data within a signal containing 
embedded watermark data according to the present 

20 invention includes: a frequency transform stop for apply- 
ing a frequency transform to the signal containing 
embedded watermark data to form an intermediate sig- 
nal; and a watermark data extraction step for extracting 
the watermark data from a predetermined frequency 

25 component of the intermediate signal. 

In one embodiment of the invention, the watermark 
data extraction stop includes determining the predeter- 
mined frequency component of the intermediate signal 
from which the watermark data is extracted based on 

30 pseudo random numbers. 

A watermark data insertion apparatus for inserting 
watermark data into an input original signal according to 
the present invention includes: a frequency transform 
section for applying a frequency transform to the origi- 

35 nal signal to form an intermediate signal; a watermark 
data insertion section for inserting the watermark data 
into a first set of frequency components of the interme- 
diate signal; and a frequency inverse transform section 
for applying an inverse frequency transform to the inter- 

40 mediate signal, the watermark data having been 
inserted into the intermediate signal, thereby obtaining 
a signal containing the watermark data embedded 
therein, wherein the watermark data insertion section 
determines the first set of frequency components of the 

45 intermediate signal into which the watermark data is 
inserted based on pseudo random numbers, and uses a 
second set of frequency components of the intermedi- 
ate signal as a reference signal, the watermark data not 
being inserted into the second set of frequency compo- 
se? nents. 

A watermark date detection apparatus for detecting 
watermark date within a signal containing embedded 
watermark data includes: an inverse-transformed 
pseudo random number generation section for applying 
55 ar\ inverse frequency transform to pseudo random num- 
bers used for determining an insertion position of the 
watermark data and complements of the Pseudo ran- 
dom numbers, respectively; a multiplication section for 
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multiplying the signal containing embedded watermark 
data by the values generated in the inverse-transformed 
pseudo random number generation section to obtain 
signal components to which the watermark data has 
been added and signal components to which the water- 5 
mark data has not been added; and an averaging sec- 
tion for deriving a difference between an average of the 
signal components to which the watermark data has 
been added and an average of the signal components 
to which the watermark date has not been added as 10 
obtained by the multiplication section, whereby the dif- 
ference is detected as the watermark data. 

In accordance with the above-described method: 
and apparatuses of the present invention, the insertion 
position and the signal amplitude of watermark data are 15 
varied in accordance with the frequency components 
and amplitude of an original signal, thereby making it 
difficult to detect the watermark data. This makes possi- 
ble the insertion of watermark data that has a large 
amplitude, thereby preventing the destruction of water- so 
mark data by an intentional addition of noise or the like. 
Since the insertion position of the watermark data is 
determined by utilizing pseudo random numbers, those 
who do not know the insertion position cannot read the 
watermark data, thereby preventing unauthorized tarn- 25 
pering with the watermark data. Furthermore, the ran- 
dom numbers used for determining the insertion 
position of the watermark data are visually weighted so 
as to render the perception of the watermark data even 
more difficult. 30 

Thus, the invention described heroin makes possi- 
ble the advantage of providing a watermark data inser- 
tion method and a watermark data detection method for 
inserting watermark data in such a manner that the 
watermark data cannot be easily detected and yet 35 
imposes minimum interference with an original signal, 
such that the watermark data is hardly visible and/or 
audible and difficult to remove by unauthorized means 
and yet easily detectable through an authorized or 
intended procedure. 40 

This and other advantages of the present invention 
will become apparent to those skilled in the art upon 
reading and understanding the following detailed 
description with reference to the accompanying figures. 

45 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram illustrating the structure 
of a watermark data insertion apparatus according 
to one embodiment of the watermark data insertion so 
method of the present invention. 

Figure 2 is a diagram showing the data arrange- 
ment of transformed coefficients of an image which 
has been broken down into two-dimensional fre- ss 
quency components by wavelet transform. 

Figure 3 is a diagram showing the position at which 



watermark date is embedded in an original signal 
coordinate system corresponding to the insertion of 
watermark data into a subband i in Figure 2. 

Figure 4 in a block diagram illustrating the structure 
of a watermark data insertion apparatus as another 
embodiment of the watermark data insertion 
method according to the present invention. 

Figure 5 is a block diagram illustrating the structure 
of a watermark data detection apparatus according 
to one embodiment of the watermark data detection 
method of the present invention. 

Figure 6 is a block diagram illustrating the structure 
of a watermark data detection apparatus as 
another embodiment of the watermark data detec- 
tion method according to the present invention. 

Figure 7 is a block diagram illustrating the structure 
of a conventional watermark data insertion appara- 
tus utilizing a spread spectrum technology. 

Figure 8 is a block diagram illustrating the structure 
of a conventional watermark data detection appara- 
tus utilizing a spread spectrum technology. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

Hereinafter, an example of the watermark data 
insertion method according to the present invention will 
be described with reference to the accompanying fig- 
ures. Figure 1 is a block diagram illustrating the struc- 
ture Of a watermark data insertion apparatus according 
to one embodiment of the watermark data insertion 
method of the present invention. The watermark data 
insertion apparatus includes a transform section 1, a 
watermark data insertion section 2, and an inverse 
transform section 3. 

The transform section 1 , which is mainly composed 
of a wavelet transform section 11 in the illustrated 
embodiment, breaks down an input image signal into 
two-dimensional frequency components. The wavelet 
transform section 11 includes a pair of filters, i.e., a 
high-pass filter and a low-pass filter (not shown), for 
separating an image signal into a high frequency com- 
ponent and a low frequency component, respectively. 
By repetitively passing the resultant high and low com- 
ponents through the same pair of filters, the wavelet 
transform section 1 1 breaks down the image signal into 
a large number of two-dimensional frequency compo- 
nents. A simplest wavelet transform section can be 
implemented by using a low-pass filter having multipli- 
cation coefficients (1/2, 1/2) and a high-pass filter hav- 
ing multiplication coefficients (1/2, -1/2) and discarding 
every other sample signal of the low frequency compo- 
nents and the high frequency components which 
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passed through the respective filters. 

Figure 2 is a diagram showing the data arrange- 
ment of transformed coefficients of an image which has 
been broken down into two-dimensional frequency com- 
ponents by wavelet transform. By a first-level wavelet 
transform, transformed coefficients in regions (sub- 
bands) LL, LH, HL, and HH, in each of which the 
number of transformed coefficients is reduced to 1/2 
along both the vertical and horizontal dimensions, are 
provided. For example, the subband LL consists of 
transformed coefficients which can be obtained by dis- 
carding every other pixel along both the vertical and 
horizontal dimensions. Therefore, restoring an image 
using the transformed coefficients from this subband 
results in an image having a 1/2 resolution of that of the 
original image. The subband LL consists of transformed 
coefficients which are derived from image components 
existing in the lower half of the frequency spectrum 
along both the vertical and horizontal dimensions. The 
subband HH consists of transformed coefficients which 
are derived from image components existing in the 
higher half of the frequency spectrum along both the 
vertical and horizontal dimensions. The subband HL 
consists of transformed coefficients which are derived 
from image components existing in the lower half along 
the vertical dimension, and in the higher half along the 
horizontal dimension, of the frequency spectrum. Simi- 
larly, the subband LH consists of transformed coeffi- 
cients which are derived from image components 
existing in the higher half along the vertical dimension, 
and in the lower half along the horizontal dimension, of 
the frequency spectrum. 

By further breaking down an image represented by 
the subband LL into higher and lower frequency compo- 
nents, subbands are obtained, each of which can be 
represented by a smaller number of transformed coeffi- 
cients. Hereinafter, the transformed coefficients (fre- 
quency components) of a subband i will be denoted as 
Bj (intermediate signal), where i is a natural number. It 
will be appreciated that the wavelet transform is simi- 
larly applicable to audio data as well. 

The watermark data insertion section 2 shown in 
Figure 1 includes a pseudo random number generation 
section 21 , a multiplication section 22, a weighting/addi- 
tion section 23, and a subtraction section 24. 

The pseudo random number generation section 21 
generates pseudo random numbers PN and outputs a 
signal PNjj, which is weighted by the pseudo random 
numbers PN, by employing an internal audio or visual 
filter (not shown). The multiplication section 22 multi- 
plies the output Bj of the transform section 1 by the out- 
put PNjj of the pseudo random number generation 
section 21 so as to output a multiplication product bj = 
Bj x PN tj . 

The weighting/addition section 23 receives water- 
mark data WWlj (representing the actual watermark), 
which is converted into watermark data Wj weighted by 
weighting coefficients corresponding to the respective 



subbands. The weighting/addition section 23 adds the 
watermark data Wj and the output signal bj of the multi- 
plication section 22 so as to output a signal Xj = bj + Wj. 
The subtraction section 24 subtracts the output signal bj 

5 of the multiplication section 22 from.the output Bj of the 
transform section 1 so as to output the subtraction result 
r,-. The inverse transform section 3 includes an inverse 
wavelet transform section 31 for receiving the, signals x t 
and Tj and transforming these signals into an image sig- 

10 nal with embedded watermark data, which is output 
from the inverse transform section 3. 

The operation of the watermark data insertion 
apparntus having the above-described structure will be 
described in more detail. A digitalized image signal is 

is transformed into signals B; of each subband i (where i 
is a natural number) by the wavelet transform section 
1 1 . The sensitivity of human eyes or ears becomes rel- 
atively high for the case where the signals Bj of a sub- 
band represent the components of a middle frequency 

so band, and relative low for the case where the signals Bj 
of a subband represent the components of a low or high 
frequency band. The pseudo random number genera- 
tion section 21 and the weighting/addition section 23 
subject each subband to a weighting in accordance with 

25 such characteristics of the human visual or auditory 
sense. 

The pseudo random number generation section 
generates pseudo random numbers PN consisting of 0 
or 1. It will be appreciated that by employing these 
30 pseudo random numbers PN as they are, it would result 
in the multiplication section 22 selecting all the sub- 
bands in an impartial or uniform manner. 

Rather, in order to prevent the unauthorized detec- 
tion of the insertion position of the watermark data 
35 WMij, the uniform pseudo random numbers PN are con- 
verted into pseudo random numbers having the oppo- 
site characteristics of those of the human visual or 
auditory sense (frequency sensitivity), by being passed 
through a filter for cutting off the components of the mid- 
40 die frequency band, for example. The pseudo random 
number generation section 21 converts the pseudo ran- 
dom number PN into pseudo random numbers PNjj 
which are weighted by means of a frequency weighting 
filter (not shown). The use of such pseudo random num- 
45 bers PNjj advantageously changes the random pattern 
from a white noise-like pattern into a pink noise-like pat- 
tern, which has less middle frequency components, 
hence making it difficult to detect. 

The multiplication section 22 multiplies the signals 
so Bj of each subband by the predetermined pseudo ran- 
dom numbers PN S j to output a multiplication product sig- 
nal b; (i.e., b; = Bj x PNjj). Thus, at least one of the 
transformed coefficients of each subband that is multi- 
plied by bit 1 of the pseudo random numbers PNjj is 
ss extracted so as to be output as the signal bj. 

Upon receiving the watermark data WWlj (of binary 
values of 0 or 1), the weighting/addition section 23 con- 
verts each value of the watermark data WMj into either 
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a positive predetermined value (when the value of the 
watermark data WMj is 1) or a negative predetermined 
value (when the value of the watermark data WMj is 0). 
Furthermore, the weighting/addition section 23 multi- 
plies the transformed watermark data by a weight Kjj 
corresponding to the subband from which the signals bj 
have been extracted so as to output the multiplication 
product as the watermark data Wj. One of respectively 
different values previously assigned to the respective 
subbands is used as the weight K;j for each subband i. 

Specifically, the value of the weight Ky is decreased 
in the middle frequency band, where humans have high 
frequency sensitivity. That is, the values of the weight 
Kjj are selected to realize the opposite characteristics of 
those of the human visual or auditory sense. As a result, 
even if the watermark data Wj is inserted in a subband 
containing middle frequency band components, the 
watermark data Wj is hardly visible (in the case of an 
image signal) and hardly audible (in the case of an 
audio signal). 

By using the pseudo random numbers PNy thus- 
weighted by the use of the frequency weighting filter 
and the weight having values corresponding to the 
respective subbands, the watermark data Wj will none- 
theless be inserted in all subbands as a result of 
processing a series of image signals representing a plu- 
rality of pictures. Therefore, even if tampering is 
attempted by passing an image signal containing 
embedded watermark data through a tow-pass filter or 
removing a portion of the signal, the watermark data will 
remain in the transformed coefficients in the middle and 
low frequency bands and therefore be detectable 
despite such tampering. 

The weighting/addition section 23 adds the water- 
mark data Wj to the signal bj and outputs a signal x s (= 
bj + Wj) containing embedded watermark data. The sig- 
nal amplitude of the watermark data Wj is selected to be 
smaller than that of the signal bj by increasing or 
decreasing the signal amplitude of the watermark data 
Wj in such a manner that the watermark data is masked 
by the signal bj to prevent unauthorized detection. 

The subtraction section 24 calculates a signal r s (= 
Bj - bj) into which the watermark data was not inserted, 
and supplies the signal r t to the inverse transform sec- 
tion 3. The inverse transform section 3 synthesizes the 
signals Xj and rj and performs a transform process at an 
inverse wavelet transform section 31 for restoring the 
original image signal. As a result of thus restoring the 
image, an image signal with embedded watermark data 
is obtained. 

The inverse wavelet transform can be achieved by, 
interpolating 0 between sample signals of each sub- 
band, passing the interpolated sample signals through 
the same high-pass filter and low-pass filter that were 
used for the wavelet transform, and repetitively deriving 
a difference between each pair of a low frequency com- 
ponent and a high frequency component output from 
the high-pass filter and low-pass filter. 
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The above-described process is repeated for the 
same watermark data, whereby image signals repre- 
senting images containing the same watermark data 
embedded therein are sequentially output. Alternatively, 

5 the addition of the same watermark data Wj to the sig- 
nal bj is performed multiple times so that a plurality of 
instances of watermark data are inserted in the signal 
Xj. In either case, a plurality of instances of the same 
watermark data will always be inserted into the image 

w signal. 

Figure 3 is a diagram showing the position at which 
watermark data is embedded in an original signal coor- 
dinate system as restored by an inverse transform 
based on the simplest wavelet transform, assuming that 

75 the watermark data was embedded at a position Xj 
among the transformed coefficients B s of a subband i 
(that lies in a region of high frequency components 
along both the horizontal direction and the vertical 
direction of the subband LL in Figure 2). The signal 

20 amplitude of the watermark data is reduced to 1/1 6 due 
to the inverse transform, and its polarity (i.e., positive or 
negative) is also changed, so that the inverse-trans- 
formed pseudo random numbers do not take binary val- 
ues but may take any real number between -1 and +1 . 

25 Figure 4 is a block diagram illustrating the structure 
of a watermark data insertion apparatus as another 
embodiment of the watermark data insertion method 
according to the present invention. As shown in Figure 
4, a multiplication section 72 directly multiplies an input 

30 image signal A ; by inverse-transformed pseudo random 
numbers which are generated by an inverse-trans- 
formed pseudo random number generation section 71 
so as to form a sample a t (i.e., a predetermined fre- 
quency component of an intermediate signal), which is 

35 output so that watermark data will be subsequently 
added thereto. Watermark data WMj is transformed into 
watermark data Wj in a base band region by an inverse 
transform section 73, and the transformed watermark 
data Wj is input to a weighting/addition section 74. The 

40 weighting/addition section 74 adds the transformed 
watermark data Wj and the sample a; so as to output a 
signal a; + Wj containing embedded watermark data. A 
subtraction section 75 calculates a sample A s - a; in 
which the watermark data is not embedded. An addition 

45 section 76 adds the output of the weighting/addition 
section 74 and the output of the substraction section 75 
so as to output an image signal containing embedded 
watermark data A s + Wj. 

fn accordance with the above apparatus, an inverse 

so transform of pseudo random numbers (indicating posi- 
tions at which watermark data is to be inserted) based 
on an inverse wavelet transform, etc., is previously cal- 
culated. Thus, the inverse-transformed pseudo random 
numbers can be used to multiply an input image signal 

55 directly, i.e., without subjecting the image signal to a 
transform process. Moreover, it becomes unnecessary 
to perform an inverse transform after the addition of 
watermark data. As a result, the cost associated with 
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the insertion of watermark data can be reduced. 

Next, one embodiment of the watermark data 
detection method according to the present invention will 
be described. Figure 5 is a block diagram illustrating the 
structure of a watermark data detection apparatus s 
according to one embodiment of the watermark data 
detection method of the present invention. The water- 
mark data detection apparatus includes a watermark 
data detection section 4, which in turn includes an 
inverse-transformed pseudo random number genera- 10 
tion section 41, a multiplication section 42, and an aver- 
aging section 43. 

The inverse-transformed pseudo random number 
generation section 41 includes a pseudo random 
number generation section, which is identical with that is 
in the watermark data insertion apparatus shown in Fig- 
ure 1, for generating pseudo random numbers PNjj and 
(1 - PNjj) and inverse transforming the pseudo random 
numbers PNjj and (1 - PNjj) by inverse wavelet trans- 
form to generate and output inverse-transformed 20 
pseudo random numbers PN^ and (1 - PNjj)'. In Figure 
5, the inverse-transformed pseudo random numbers 
PNij f and (1 - PNjj)' are shown as Nj and M i( respec- 
tively; Nj is an inverse wavelet transform of the pseudo 
random numbers PNjj used for the determination of 25 
watermark data insertion position; and Mj is an inverse 
wavelet transform of the complement of the pseudo ran- 
dom numbers PNjj. 

Upon receiving an image signal P containing 
embedded watermark dots, the multiplication section 42 30 
multiplies the image signal P by the inverse-transformed 
pseudo random numbers Nj and Mj to output a signal Xj* 
(= P x PNjjO and a signal r{ (=Px(1- Pl^y). 

By using these inverse-transformed pseudo ran- 
dom numbers Nj and Mj, it becomes unnecessary to 55 
subject image data containing watermark data to a 
wavelet transform in order to achieve detection of the 
watermark date. As a result, the time and cost required 
for detection are reduced. 

The resultant signal Xj' is a signal containing 40 
embedded watermark data. The resultant signal iy is a 
signal not containing the watermark data embedded 
therein. 

The signals Xj' and r{ are input to an averaging sec- 
tion 43, where a cumulative sum LxP and a cumulati /e 45 
sum ZrY are calculated for the signals Xj' and iy, respec- 
tively. A difference therebetween (Lxi* - Lri*) is calcu- 
lated. 

The value of the above difference equals x{- iy = 
(bj 1 + WMj') - iy. Since the signals b s are generated by so 
nondiscriminatory selection based on random numbers, 
an average of a multitude of bj f and an average of a mul- 
titude of iy will satisfy bj 1 = iy (because the population of 
bj* and the population of iy are identical). Thus, the 
above subtraction leaves WMj* (= (bj* + WMj') - rj'), i.e., 55 
the watermark data. Based on the signs (i.e., positive or 
negative) of this value, the value of the watermark data 
WMj can be obtained. 



When calculating an average of the signals x{ con- 
taining embedded watermark data, the average of 
higher frequency components receives a weight which 
is smaller than the weights for other average values. As 
a esult, even if an unauthorized third party attempts to 
destroy or render undetectable the watermark data, or if 
the size or the position of the signal containing embed- 
ded watermark data is changed, or if noise is admixed in 
the i ;igh frequency components, such influences can be 
effectively eliminated. 

Figure 6 is a block diagram illustrating the structure 
of a watermark data detection apparatus as another 
embodiment of the watermark data detection method 
according to the present invention. As shown in Figure 
6, a transform section 81 receives an image signal P 
containing embedded watermark data and performs a 
wavelet transform to transform the image signal P into 
transformed coefficients of each subband. This process 
i? in the opposite direction with respect to the process 
performed by the inverse transform section 3 in Figure 
The signal (i.e., intermediate signal) which results 
tiom this process is equal to Xj + r x in the embodiment 
described with reference to Figure 1 . A multiplication 
section 83 multiplies the signal x t + Tj by pseudo random 
numbers generated by a pseudo random number gen- 
eration section 82 (which are the same pseudo random 
numbers as those used in the embodiment described 
with reference to Figure 1) so as to extract a signal Xj 
containing embedded watermark data. The signal x s 
consists of the original signal bj with the watermark data 
Wj added thereto. The averaging section 85 derives an 
average of a plurality of signals x s containing embedded 
watermark data to give a sum (B + W) of an average B 
of the original signals bj and an average W of the water- 
mark data. 

Only the signal rj (= Bj - bj) in which the watermark 
data was not embedded is output from a subtraction 
section 84. An averaging section 86 derives on average 
of a plurality of signals r lt thereby giving an average B of 
the original signals. A subtraction section 87 subtracts 
the average B from the average containing the water- 
mark data (B + W), thereby obtaining the watermark 
data W. 

Since the input signal is subjected to wavelet trans- 
form in the apparatus shown in Figure 6, the same 
pseudo random numbers as those used in the above- 
described watermark data insertion apparatuses can be 
used. 

Although a wavelet transform section is exemplified 
as a transform section in the above- illustrated embodi- 
ments, the some effects will be provided by employing a 
frequency transform means such as Hadamard trans- 
form, discrete cosine transform, Fourier transform, Kar- 
hunen-Lo6ve transform, or the like. Pseudo random 
numbers can be generated by using linear feedback 
shift registers. Alternatively, previously calculated 
pseudo random numbers can be stored in a memory or 
the like for use in the process of the present invention. 
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Various sections illustrated in the embodiments can 
be implemented as computer programs or dedicated 
digital circuits. 

As described above, the watermark data insertion 
method and watermark data detection method accord- s 
ing to the present invention make it possible to insert 
watermark data in a manner such that the watermark 
data is very difficult to detect by unauthorized third par 
ties while being easily detectible by an authorized per 
son, e.g., one who inserted the watermark data. Thus 10 
the present invention makes possible the establishment 
of infringement of a copyright of copyrightable works 
and/or prevention of illegal copying of such works in the 
field of digital contents containing images and/or 
sounds. 15 

In accordance with one embodiment of the present 
invention, a wavelet transform encompasses the pixel 
information of the entire single frame regardless of the 
subband that is being used. Therefore, watermark data 
can be easily embedded in the entire image through 20 
one watermark data insertion process. The method of 
the present invention also prevents block distortion, 
which is typically observed in DCT transform. 

In accordance with one embodiment of the present 
invention, watermark data is inserted with its amplitude 25 
being adjusted in accordance with the luminance values 
of pixels, whereby the watermark data can be better 
concealed within the image. 

In accordance with one embodiment of the present 
invention, watermark data can be inserted in positions 30 
in the pixel frequency space where the watermark data 
will not be easily noticeable. 

In accordance with one embodiment of the present 
invention, the DC component of the watermarked data 
is not changed after insertion, thereby improving the 35 
signal to noise ratio in the averaging process during 
detection. 

In accordance with one embodiment of the present 
invention, it is unnecessary to detect watermark data by 
referring to a reference image, thereby reducing the bur- 40 
den on the detection system. 

In accordance with one embodiment of the present 
invention, watermark data can be detected even after 
processing a signal with a high-cut filter. 

Various other modifications will be apparent to and 45 
can be readily made by those skilled in the art without 
departing from the scope and spirit of this invention. 
Accordingly, it is not intended that the scope of the 
claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly so 
construed. 

Claims 

1. A watermark data insertion method for inserting 55 
watermark data into an input original signal, com- 
prising: 



a frequency transform step for applying a fre- 
quency transform to the original signal to form 
an intermediate signal; 

a watermark data insertion stop for inserting 
the watermark data into a first set of frequency 
components of the intermediate signal; and 
a frequency inverse transform step for applying 
an inverse frequency transform to the interme- 
diate signal, the watermark data having been 
inserted into the intermediate signal, thereby 
obtaining a signal containing the watermark 
data embedded therein, 
wherein the watermark data insertion step 
comprises: 

determining, based on pseudo random 
numbers, the first set of frequency compo- 
nents of the intermediate signal into which 
the watermark data is inserted; and 
using a second set of frequency compo- 
nents of the intermediate signal as a refer- 
ence signal, the watermark data not being 
inserted into the second set of frequency 
components. 

2. A watermark data insertion method according to 
claim 1 , wherein the frequency transform applied in 
the frequency transform step is a wavelet trans- 
form, 

and the inverse frequency transform applied 
in the frequency inverse transform step is an 
inverse wavelet transform. 

3. A watermark date insertion method according to 
claim 1, wherein the watermark data insertion step 
comprises: 

a weighting step for increasing or decreasing a 
signal amplitude of the watermark data in 
accordance with on amplitude of the first set of 
frequency components of the intermediate sig- 
nal into which the watermark data is inserted. 

4. A watermark data insertion method according to 
claim 1, wherein the watermark data insertion step 
comprises determining the first set of frequency 
components of the intermediate signal into which 
the watermark data is inserted by using pseudo 
random numbers having frequency-dependent 
weights. 

5. A watermark data insertion method according to 
claim 1 , wherein the watermark data insertion step 
comprises: 

a weighting step for increasing or decreasing a 
signal amplitude of the watermark data in 
accordance with a frequency of the first set of 
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frequency components of the intermediate sig- 
nal into which the watermark data is inserted. 

6. A watermark data insertion method according to 
claim 1 . wherein the watermark data insertion step s 
comprises converting binary data representing the 
watermark data into predetermined positive and 
negative values, and thereafter inserting the con- 
verted binary data into the first set of frequency 
components of the intermediate signal. w 

7. A watermark data insertion method for inserting 
watermark data into an input original signal, com- 
prising: 

15 

determining a position at which the watermark 

data is to be inserted in accordance with 

pseudo random numbers which have been 

subjected to an inverse transform; 

subjecting the watermark data to an inverse 20 

transform; 

increasing or decreasing a signal amplitude of 
the inverse-transformed watermark data in 
accordance with an amplitude of the original 
signal; and 25 
inserting the inverse-transformed watermark 
data with the increased or decreased signal 
amplitude into the original signal. 

8. A watermark data detection method for detecting 30 
watermark data within a signal containing embed- 
ded watermark data, comprising: 

an inverse-transformed pseudo random 
number generation step for applying an inverse 35 
frequency transform to pseudo random num- 
bers used for determining an insertion position 
of the watermark data and complements of the 
pseudo random numbers, respectively; 
a multiplication step for multiplying the signal ao 
containing embedded watermark dots by the 
values generated in the inverse-transformed 
pseudo random number generation step to 
obtain signal components to which the water- 
mark data has been added and signal compo- as 
nents to which the watermark data has not 
been added; and 

an averaging step for deriving a difference 
between an average of the signal components 
to which the watermark data has been added so 
and an average of the signal components to 
which the watermark data has not been added 
as obtained in the multiplication step, whereby 
the difference is detected as the watermark 
data. ss 

9. A watermark data detection method according to 
claim 8, wherein the averaging step comprises 



16 

deriving the average of the signal components to 
which the watermark data has been added by 
applying a weight to an average of high frequency 
components among the signal components to 
which the watermark data has been added, the 
weight being smaller than weights applied to aver- 
ages of frequency components other than the high 
frequency components. 

10. A watermark data detection method for detecting 
watermark data within a signal containing embed- 
ded watermark data, comprising: 

a frequency transform step for applying a fre- 
quency transform to the -signal containing 
embedded watermark data to form an interme- 
diate signal; and 

a watermark data extraction step for extracting 
the watermark data from a predetermined fre- 
quency component of the intermediate signal. 

11. A watermark data detection method according to 
claim 10, wherein the watermark data extraction 
step comprises determining the predetermined fre- 
quency component of the intermediate signal from 
which the watermark data is extracted based on 
pseudo random numbers. 

12. A watermark data insertion apparatus for inserting 
watermark data into an input original signal, com- 
prising: 

a frequency transform section for applying a 
frequency transform to the original signal to 
form an intermediate signal; 
a watermark data insertion section for inserting 
the watermark data into a first set of frequency 
components of the intermediate signal; and 
a frequency inverse transform section for 
applying an inverse frequency transform to the 
intermediate signal, the watermark data having 
been inserted into the intermediate signal, 
thereby obtaining a signal containing the water- 
mark data embedded therein, 
wherein the watermark data insertion section 
determines the first set of frequency compo- 
nents of the intermediate signal into which the 
watermark data is inserted based on pseudo 
random numbers, and uses a second set of fre- 
quency components of the intermediate signal 
as a reference signal, the watermark data not 
being inserted into the second set of frequency 
components. 

1 3. A watermark data detection apparatus for detecting 
watermark data within a signal containing embed- 
ded watermark data, comprising: 
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an inverse-transformed pseudo random 
number generation section for applying an 
inverse frequency transform to pseudo random 
numbers used for determining an insertion 
position of the watermark data and comple- $ 
ments of the pseudo random numbers, respec- 
tively; 

a multiplication section for multiplying the sig- 
nal containing embedded watermark data by 
the values generated in the inverse-trans- 10 
formed pseudo random number generation 
section to obtain signal components to which 
the watermark data has been added and signal 
components to which the watermark data has 
not been added; and is 
an averaging section for deriving a difference 
between an average of the signal components 
to which the watermark data has been added 
and an average of the signal components to 
which the watermark data has not been added 20 
as obtained by the multiplication section, 
whereby the difference is detected as the 
watermark data. 
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